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My  name  is  Andy  McKinley  and  welcome  to  the  presentation  entitled,  “Tactile 
Cueing  for  Target  Acquisition  and  Identification.”  I  would  like  to  acknowledge 
Dr.  Jennie  Gallimore  for  her  valuable  guidance  on  this  project  and  my  other 
coauthors  for  their  contribution  to  data  collection  and  the  final  report. 


Background 


•  Most  information  displayed  visually  in  cockpit  -  can 
lead  to  over  saturation  of  this  modality 

—  Other  modalities  of  presenting  info,  (such  as  tactile) 
should  be  explored 

•  Sense  of  touch  can  provide  large  amount  of  info. 

—  has  10,000  parallel  channels  capable  of  responding 
to  stimulus  interruptions  as  short  as  10  ms 
(Cholewiak  &  Collins,  1991),  (Sherrick  &  Cholewiak 
1986) 


This  research  was  conducted  because  currently,  almost  all  information  in  aircraft 
cockpits  is  displayed  to  the  pilot  visually.  Currently,  more  and  more  information 
is  being  crammed  into  already  limited  real  estate.  As  a  consequence,  many 
visual  displays  are  becoming  menu  based  or  multifunction  which  forces  the  pilot 
to  push  several  buttons  to  find  the  information  that  he/she  needs.  In  addition, 
the  visual  system  is  prone  to  oversaturated  and  attention  narrowing.  In  the  high 
workload  flight  environment,  this  can  cause  the  pilot  to  negate  instrument 
checks  or  to  miss  critical  information.  Thus,  other  modalities  of  presenting 
information  should  be  investigated.  The  sense  of  touch  was  used  in  this  study 
due  to  the  fact  that  it  is  capable  of  providing  a  large  amount  of  information.  It 
has  10,000  parallel  channels  that  are  capable  of  responding  to  stimulus 
interruptions  as  short  as  10ms. 


Background 


—  6  different  tactile  receptors  that  have  been  identified  and 
characterized  (Cholewiak  &  Collins,  1991),  (Cholewiak,  1999 

•  two  housed  in  hairy  skin 

•  two  only  found  in  hairless  (glabrous)  skin 

•  two  found  in  both 

—  These  receptors  differ  in  adapting  rate,  receptive  field  size, 
sensitive  frequency  range,  and  the  sensation  that  they  evoke 
(Cholewiak  &  Collins,  1991),  (Sherrick  &  Cholewiak  1986), 
(Wainstein  1968) 

—  Example:  Pacinian  corpuscles  create  sensation  of 
vibration/tickle;  Ruffini  endings  provide  sensations  of  stretch, 
shear,  and  tension 


There  are  6  different  tactile  receptors  that  have  been  identified  and  the  are  not 
found  in  all  areas  of  the  skin.  For  example,  two  are  found  in  only  hairy  skin,  two 
others  are  only  found  in  hairless  skin,  and  two  can  be  found  in  both.  These 
receptors  differ  in  adapting  rate,  receptive  field  size,  sensitive  frequency  range, 
and  the  sensation  that  they  evoke.  For  example,  Pacinian  corpuscles  create  the 
sensation  of  vibration/tickle;  Ruffini  endings  provide  sensations  of  stretch,  shear, 
and  tension. 


Background 


•  Tactile  vests: 

•  Tactile  Situational  Awareness  System  (TSAS)- 
Navy  (McGrath,  et  al.  2004) 

—  Provides  spatial  awareness  information  to 
pilots 

—  Accepts  data  from  various  sensors  and 
displays  this  information  via  vibrators  or 
tactors  integrated  into  flight  gear 

—  24  tactor  array  (8  columns  and  3  rows  around 
the  torso)  and  4  electromagnetic  tactors  (2  on 
the  shoulder  and  2  on  the  ventral  thigh  area). 


30*  ROLL  LEFT 


Tactile  displays  are  not  new  technology.  The  navy  has  been  developing  and 
testing  a  tactile  display  called  the  Tactile  Situational  System,  or  TSAS.  The 
purpose  of  this  vest  is  to  provide  information  about  the  orientation  of  the  aircraft 
to  the  pilot.  The  system  accepts  data  from  onboard  sensors  and  then  displays 
this  information  through  vibrating  tactors  on  the  torso.  The  current  configuration 
is  a  24-tactor  array  (8  columns  and  3  rows  around  the  torso).  It  also  contains  4 
strong  electromagnetic  tactors  -  2  on  the  seat  and  two  on  the  shoulders.  A 
photo  of  the  vest  can  be  seen  at  the  bottom  of  this  slide. 


Background 


•  Pilots  demonstrated  improved  control  of  aircraft  during  complex  flight 
conditions  with  TSAS  (McGrath  et.  al.  2004) 

•  Shown  to  increase  Situation  Awareness  (SA)  (Rupert  et.  Al.  1996) 

—  This  provides  opportunity  to  devote  more  time  to  other  instruments 
and  systems  when  flying  in  task  saturated  conditions 

•  Reduced  pilot  workload 

•  Shown  to  aid  helicopter  pilots  in  hover  (McGrath  et.  al.  2004) 

—  Reduced  drift 

•  Can  provide  wide  variety  of  information  (Chiasson  et.  Al,  2002) 

—  attitude,  altitude,  velocity,  navigation,  acceleration,  threat  location, 
targets 


Some  of  the  outcomes  of  TSAS  research  are  displayed  here.  The  navy  has 
been  able  to  show  that  pilots  demonstrated  improved  control  of  the  aircraft 
during  complex  conditions,  increased  situational  awareness  of  the  pilot,  reduced 
workload,  and  aided  helicopter  pilots  in  hover  by  reducing  drift.  In  addition, 
tactile  displays  are  capable  of  displaying  a  wide  range  of  information  to  the  pilot 
besides  orientation  information,  including  altitude,  closure  rate,  velocity, 
acceleration,  threat  location,  and  the  type  of  target. 


Objective  |Pj 
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Goal: 

Improve  methods  of  displaying  information  about  target 
aircraft  to  a  pilot  through  tactile  displays  on  the  torso 

Specific  Aims: 

•  Determine  optimal  azimuth  resolution  of  the  tactile  display 

•  Establish  optimal  method  of  presenting  target  aircraft  altitude 
information 

•  Assess  whether  coding  symbology  to  identify  types  of  aircraft 
is  useful  and  effective 


The  goal  of  this  particular  study  was  to  improve  the  methods  of  displaying 
information  about  targets  to  a  pilot  through  torso  tactile  displays.  Specifically, 
this  study  was  designed  to  optimize  the  resolution  of  the  display  in  the  azimuth 
plane,  establish  the  optimal  method  of  displaying  aircraft  altitude  information, 
and  assess  whether  coding  symbology  to  identify  the  type  of  target  is  useful 
and/or  effective. 


A  total  of  19  subjects  participated  in  this  study;  12  male,  7  female.  All  subjects 
had  to  meet  three  physical  criteria  to  participate.  Their  normal  or  corrected 
vision  had  to  be  at  least  20/40,  waist  size  had  to  be  28  to  36  inches,  and  chest 
size  had  to  be  between  34  and  40  inches.  The  size  requirement  was  due  to  the 
size  limitations  of  the  vest. 
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Apparatus 


Hardware: 

—  Standard  flight  vest  with  36  pager  motors 

•  Standard  laboratory  power  supply  or 
batteries 

•  National  Instruments  6513  high-current 
digital  output  card 

—  CyVisor  HMD 

—  Flock  of  Birds  head-tracker  (HT) 

—  Headphones  (mount  for  HT  sensor) 

—  Dell  Premium  330 

•  Runs  flight  sim  graphics 

—  Lab  computer 

•  Runs  tactile  vest  control  software  and 
displays  GUI  for  PI 


The  tactile  vest  used  in  this  study  was  a  standard  combat  edge  flight  vest 
instrumented  with  36  pager  motors  or  tactors.  The  tactors  were  arranged  in 
three  rows  and  12  columns.  A  CyVisor  provided  the  visual  display  to  the 
subjects,  the  head  tracker  allowed  the  subjects  to  look  around  in  the  virtual 
environment  and  two  computers  were  used  to  run  the  visual  software  and  collect 
data. 


Apparatus 


•  Software 

— X-plane  flight 
simulator  - 
performance  task 

—  LabVIEW  - 
controls  for  tactile 
vest 


The  flight  simulator  used  was  a  modified  version  of  x-plane  and  the  tactor  vest 
was  controlled  through  custom  software  developed  with  LabView. 


Experimental  Design 

•  3x2x2  full-factorial 

•  Mixed  subject  design 

•  Repeated  measures 

—  12  trials  per  subject 
—  Latin  square  used  to  randomize  order  of  conditions  presented 
—  Subjects  were  randomly  assigned  a  subject  number. 

•  Dependent  measures 

—  Time  to  acquire  each  target 
—  Number  of  errors  made 


The  experimental  design  was  a  repeated  measures  three  by  two  by  two  full 
factorial.  There  were  12  data  trail  per  subject  and  a  Latin  Square  design  was 
used  to  randomize  the  order  of  the  treatment  conditions.  Subjects  were 
randomly  assigned  a  subject  number.  The  dependent  measures  collected 
included  the  time  to  acquire  each  target  and  the  number  of  errors  made. 


Experimental  Design 


•  Tactor  resolution  (within  subjects) 

—  Level  1:12  available  tactors 

—  Level  2:  8  available  tactors 
—  Level  3:  4  available  tactors 

•  Target  type  coding  (within  subjects) 

—  Level  1:  single  code 

—  Level  2:  separate  code  for  each  target  type 


ii 


The  first  independent  variable  was  the  tactor  resolution.  There  were  three 
levels,  which  included  12,  8,  and  4  tactors.  The  second  factor  was  the  coding 
for  the  type  of  target.  Either  a  single  code  was  presented  regardless  of  the  type 
of  target,  or  a  separate  code  was  presented  relating  each  target  to  a  different 
tactile  pattern.  Both  variables  were  within  subjects. 


Experimental  Design 


Elevation  presentation  (between  subjects) 

—  Level  1 :  5  elevation  settings 

•  >40°  above  =  top  tactor  activated 

•  6-39°  above  =  top  and  middle  tactors  activated 

•  ±5°  =  middle  tactor  activated 

•  6-39°  below  =  middle  and  lower  tactors  activated 

•  >40°  below  =  lower  tactor  activated 

—  Level  2:  3  elevation  settings 

•  >6°  =  top  tactor  activated 

•  ±5°  =  middle  tactor  activated 

•  <6°  =  lower  tactor  activated 


The  final  factor  was  the  method  of  elevation  presentation,  which  was  a  between 
subjects  variable.  There  were  either  3  or  five  settings  for  the  elevation  of  the 
target  presentation,  which  are  listed  here.  Essentially,  level  one  provided  two 
more  degrees  of  resolution  to  the  subjects  than  level  2. 


Experimental  Design 


•  3  types  of  aircraft  were  displayed 

•  No  more  than  2  types  of  aircraft  at  one  time 

•  No  more  than  2  of  any  one  type  at  one  time 


Type  of  Aircraft  j 

Condition 

Enemy 

Unknown 

Friendly 

1 

1 

0 

2 

2 

1 

2 

0 

3 

2 

0 

1 

4 

2 

1 

0 

5 

0 

2 

1 

6 

0 

1 

2 
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The  task  used  in  this  experiment  used  three  different  types  of  aircraft,  enemy, 
unknown,  and  friendly.  The  type  of  aircraft  was  randomized  under  the 
constrictions  that  no  more  than  two  types  of  aircraft  could  be  displayed  in  any 
one  trial,  and  no  more  than  two  of  any  one  type  of  aircraft  could  be  displayed  in 
any  one  trial.  For  example,  there  could  never  be  3  enemy  targets  displayed 
simultaneously.  This  left  six  possible  configurations  which  are  displayed  in  this 
chart.  The  numbers  represent  the  number  of  that  particular  aircraft  to  be 
displayed  in  that  trial. 


Stimuli 

•  Computer  generated  simulated  flight  environment 

—  subject’s  virtual  aircraft  traveled  along  a  set  flight  path  at  an  altitude  of 
7,000 

—  A  mil  standard  1787B  heads  up  display  (HUD)  overlaid  the  scenery 

—  3  targets  per  presentation  (enemy,  friendly,  or  unknown) 

—  Enemy  =  MIG-29  Fighter  Jet 
—  Unknown  =  British  Airways  Jetliner 
—  Friendly  =  UH-€0  Blackhawk  Helicopter 

•  Target  Tactile  Patterns 
—  Enemy  -  3  short  pulses 

—  Unknown  - 1  long  pulse  followed  by  2  short  pulses 
—  Friendly  -  2  long  pulses 
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The  visual  stimuli  consisted  of  a  computer  generated  flight  environment  that 
placed  the  subject  inside  the  cockpit  off  an  F-22  fighter  jet.  The  subjects’  aircraft 
was  held  on  a  fixed  flight  path  that  maintained  a  constant  altitude  of  7,000  feet. 

In  addition,  a  mil  sttandard  1787B  heads  up  display  was  overlaid  on  the  visual 
scenery.  Three  target  aircraft  were  presented  simultaneously  each  trial.  The 
enemy  aircraft  was  a  Mig-29,  the  unknown  was  a  British  Airways  jetliner,  and  the 
friendly  target  was  a  UH-60  Blackhawk  helicopter.  These  targets  were  chosen 
due  to  the  fact  that  they  were  easily  visually  discernable.  Each  type  of  aircraft 
had  a  different  tactile  pattern  associated  with  it.  The  pattern  for  the  enemy 
aircraft  was  3  short  pulses,  the  unknown  was  1  long  pulse  followed  by  2  short 
pulses,  and  the  friendly  aircraft  was  two  long  pulses.  These  patterns  were 
chosen  to  give  the  more  dangerous  aircraft  a  more  urgent  tactile  pattern. 


Stimuli 


•  Each  target’s  position  was 
random  (azimuth,  elevation) 

—  Distance  from  subject  set 
to  constant  (500  ft) 

•  Targets  do  not  move  relative 
to  subject 


IS 


Each  target’s  position  was  random  in  azimuth  and  elevation  angle.  The 
elevation  angles  were  confined  +70  degrees  due  to  the  fact  that  the  head  tracker 
failed  to  operate  properly  out  of  this  range.  In  addition,  the  azimuth  angle  was 
confined  to  +150  degrees  due  to  the  fact  that  is  was  difficult  for  the  subjects  to 
twist  their  bodies  a  full  180  degrees.  The  targets  were  set  at  a  constant  distance 
of  500  feet  from  the  subject  so  that  each  target  would  have  the  same  visual 
acuity.  In  addition,  the  targets  did  not  move  relative  to  the  subject’s  aircraft. 


Procedure 


•  Training 

—  Received  task  instructions  for  the  training 
session 

—  Familiarized  subject  with  tactor 
placement/sensation 

—  Performed  the  task  in  ail  6  conditions  twice 

•  Data  Collection 

—  Conducted  at  WSU-Russ-Room  250 


During  training,  the  subjects  read  written  instructions  for  the  task  based  on 
group  they  were  assigned.  The  investigator  then  familiarized  the  subject  with 
the  tactor  placement  and  sensation  by  exciting  tactors  in  a  variety  of  locations 
and  naming  that  location  for  the  subject.  Each  subject  then  performed  all  six 
conditions  twice.  The  data  collection  was  performed  at  Wright  State  University’ 
Russ  engineering  center,  room  250. 
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Procedure 


•  Task  Execution 

—  Set  of  three  target  aircraft  (in  one  of  6  treatment 
combinations)  appeared  in  randomized  locations 

—  Tasked  with  visually  acquiring  each  target  in  the 
order  of  all  enemies,  all  unknowns,  and  then  all 
friendlies. 

•  Out  of  sequence  =  Error 

•  Successful  acquisition:  Align  center  “x”  on  HUD 
with  center  of  target  aircraft  for  two  seconds. 

•  Target  and  tactile  sensation  related  to  that 
aircraft  disappeared  after  visual  acquisition 

•  Task  timed  out  after  60  seconds  =  Error 


For  each  task  execution,  a  set  of  three  targets  appeared  in  randomized 
locations.  The  subjects  were  tasked  with  visually  acquiring  the  targets  in  the 
order  of  all  enemies,  all  unknowns,  and  then  all  friendlies  as  quickly  and 
accurately  as  possible.  If  the  aircraft  were  acquired  out  of  sequence,  an  error 
was  reported  in  the  data  output.  To  successfully  acquire  a  target,  the  subject 
had  to  align  the  center  “x”  on  the  HUD  with  the  center  of  the  target  for  two 
consecutive  seconds.  The  target  and  the  corresponding  tactile  cue  would 
disappear  after  a  successful  acquisition.  The  task  timed  out  after  60  seconds. 
IN  the  event  this  occurred,  the  remaining  aircraft  would  be  reported  as  an  error. 


Procedure 


•  Subjects  completed  two  presentations  for  each 
treatment  combination 

—  Time  to  acquisition  and  error  rate  data  were 
averaged  across  subjects  for  each  condition 

*  In  addition,  subjects  were  asked  how  often  they  play 
video  games 

—  Separated  into  two  groups 

•  Video  Game  Players  (VGPs)  -  Play  minimum  of  2- 
3  hours  per  week 

•  Non-Video  Game  Players  (NVGPs)  -  Never  play 


Subjects  completed  two  presentations  for  each  treatment  combination.  The 
dependent  measures  recorded  in  the  data  file  were  the  time  to  acquistion  for 
each  of  the  targets  and  the  number  of  errors  for  each  trial.  These  data  were 
averaged  across  subjects  for  each  condition.  Following  the  data  collection, 
subjects  were  asked  how  often  they  play  video  games.  Based  on  their 
responses,  they  were  separated  into  two  groups:  Video  Game  Players  and  Non- 
Video  game  Players.  Video  game  players  played  an  average  of  two-three  hours 
per  week,  while  non-video  game  players  did  not  play  at  all. 


This  slide  discusses  the  results  for  the  video  game  player  group.  The  visual 
acquisition  time  was  averaged  across  all  three  targets  for  each  presentation  of 
the  task.  The  was  a  statistically  significant  main  effect  of  azimuth  resolution  on 
acquisition  time.  Tukey’s  test  was  then  employed  to  determine  the  resolution 
that  produced  the  lowest  acquisition  time.  The  acquisition  time  for  the  12  tactor 
condition  was  significantly  less  than  for  the  8  or  4  tactor  resolution.  The  p-value 
was  0.0082. 
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Visual  Acquisition  Time  (NVGPs) 

—  Averaged  across  all  three  targets  for  each  presentation  of  the  task 

—  Significant  main  effect  of  azimuth  resolution  on  Acquisition  Time 

—  Tu key’s  test  used 

•  Times  were  significantly  shorter  for  the  8  tactor  treatment  level 

•  Azimuth  p=0.0081 


Error  Rate 

—  Significant  main  effect  of  pattern  on  Error  Rate 

—  Tukey’s  test  used 

•  Errors  were  significantly  greater  for  the  “No  Tactile  Pattern”  level 

•  Azimuth  p=0.0066 


The  results  for  the  non-video  game  player  group  were  much  different  than  the 
results  for  the  video  game  player  group.  The  data  was  averaged  as  before  and 
again,  the  azimuth  resolution  showed  a  significant  main  effect  on  acquisition 
time.  Tukey’s  test  showed  that  the  acquisition  times  were  significantly  shorter 
for  the  8-tactor  resolution  (p-value  of  0.0081).  In  addition,  there  was  a 
significant  effect  of  tactile  pattern  on  the  error  rate.  The  condition  in  which  a 
different  tactile  pattern  was  NOT  administered  corresponding  to  the  different 
target  types  produced  fewer  errors  than  the  condition  where  the  tactile  patterns 
indicating  the  type  of  target  were  given. 


Discussion 


•  12  tactors  in  the  azimuth  plane  produced  better 
performance  (lower  acquisition  time)  in  visual  target 
acquisition  than  4  or  8  for  VGPs 

•  8  tactors  in  the  azimuth  plane  produced  better 
performance  (lower  acquisition  time)  in  visual  target 
acquisition  than  12  or  4  for  NVGPs 

—  NVGPs  -  Errors  were  significantly  greater  for  the 
“No  Tactile  Pattern"  level 

—  Indicates  NVGPs  performed  better  with  less  info. 

•  Although  the  coding  pattern  was  not  significant,  it  was 
approaching  significance  (p=0.0603) 

—  Perhaps  would  be  significant  with  more  data 


The  results  showed  that  the  video  game  player  group  performed  better  with  the 
highest  azimuth  resolution.  The  non-video  game  players  performed  better  with 
a  lower  azimuth  tactile  resolution.  Neither  group  performed  well  in  the  4-tactor 
configuration  which  indicates  that  four  tactors  in  the  azimuth  plane  is  not 
adequate  to  provide  target  location  information.  Furthermore,  the  non-video 
game  players  had  significantly  fewer  errors  when  given  the  type  of  aircraft  was 
not  coded  with  a  tactile  pattern.  This  indicates  that  the  non-video  game  player 
group  performed  better  with  less  information  and  less  tactors.  Anecdotally, 
many  of  the  non-video  game  players  found  the  tactile  patterns  overwhelming  or 
confusing  and  found  it  difficult  to  interpret  the  signals.  In  addition,  many  of  them 
found  12  tactors  of  azimuth  resolution  was  too  much,  whereas  many  of  the  video 
game  players  preferred  the  12-tactor  arrangement. 


Conclusions 


•  Due  to  the  fact  that  both  VGPs  and  pilots  are 

accustomed  to  responding  to  multiple  stimuli  in  a  high 
workload  environment,  it  is  likely  the  performance  of 
the  VGP  group  is  more  representative  of  a  pilot’s 
performance. 

—  Therefore,  the  conclusions  of  this  study  are  that  12 
tactors  per  row  (azimuth)  should  be  the  minimum 
requirement  for  torso  tactile  displays. 

—  Tactile  patterns  were  not  useful  to  the  subjects  in 
this  experiment,  however  different  or  a  fewer 
number  of  patterns  may  lead  to  different  results. 


It  is  anticipated  that  the  video  game  player  group’s  performance  is  more  closely 
linked  with  a  pilot’s  performance  than  the  non-video  game  players.  The  reason 
for  this  is  that  both  pilots  and  video  game  players  must  constantly  scan  their 
environment  and  respond  to  multiple  stimuli.  This  functional  difference  between 
the  group  of  subjects  leads  to  the  conclusion  that  the  results  for  the  VGP  group 
should  be  used  in  the  design  of  tactile  displays  for  pilots.  Twelve  tactors  per  row 
should  be  the  minimum  requirement  for  torso  tactile  displays.  Although  tactile 
patterns  were  not  useful  to  the  subjects  in  this  experiment,  different  or  a  fewer 
number  of  patterns  may  lead  to  better  performance. 


Recommendations  for  Future 
Studies 


•  Compare  age  groups 

—  Tactile  symbology  may  be  more  intuitive  for  younger  subjects 
based  on  video  game  experience 

•  Compare  visual  only  to  visual  with  tactile  cues 

•  Limit  azimuth  angle  to  +130  degrees 

—  limit  difficulty  of  turning  in  chair 

•  More  Realistic  Simulation 

—  Fly  aircraft -turn  aircraft,  not  torso 
—  No  acquisition  of  friendly  aircraft 
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The  recommendations  for  the  future  studies  are  to  compare  age  groups.  It  is 
anticipated  that  younger  subjects  may  adapt  better  to  tactile  symbology  due  to 
video  game  experience.  In  addition,  future  studies  should  compare  performance 
visual  only  cues  to  performance  with  tactile  and  visual  cues.  If  completed  under 
a  similar  design,  the  azimuth  resolution  should  be  limited  to  plus  or  minus  130 
degrees  in  future  studies  due  to  difficulties  experienced  in  turning  in  a  stationary 
chair.  It  would  also  be  ideal  to  have  the  subjects  fly  the  aircraft  toward  the 
targets  to  acquire  them  as  they  would  in  the  operational  environment.  Finally, 
the  acquisition  of  friendly  targets  should  be  removed  from  the  design.  Perhaps 
with  only  two  different  tactile  patterns,  the  symbology  would  have  been  more 
useful. 


More  Recommendations 


•  Investigate  other  patterns  for  target  coding 

•  Determine  maximum  number  of  different  patterns  that 
can  be  determined  simultaneously 


Other  recommendations  are  to  investigate  other  patterns  of  coding  targets  and 
to  determine  that  maximum  number  of  different  patterns  that  can  be  identified 
simultaneously. 
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